Apparatus and method for signal processing

ABSTRACT

A signal processing apparatus includes a frequency detector configured to receive a user input including at least one of a vibration input and a user voice, vibrate in response to the received user input, and detect a frequency of the received user input, based on the vibration, and a processor configured to determine a type of the user input received by the frequency detector, based on the frequency detected by the frequency detector, and perform a function corresponding to the user input of the determined type.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2021-0072967, filed on Jun. 4, 2021,and Korean Patent Application No. 10-2021-0171204, filed on Dec. 2,2021, in the Korean Intellectual Property Office, the disclosures ofwhich are incorporated by reference herein in their entirety.

BACKGROUND 1. Field

The disclosure relates to an apparatus and method for signal processing.

2. Description of Related Art

A vibration sensor that is installed in various electronic devices andsenses a vibration input and an acoustic sensor that is installed invarious electronic devices and detects a user's voice are increasinglybeing used. However, individual sensors are used to sense differenttypes of user inputs, and thus process costs, complexity, and powerconsumption increase. The complexity of calculations also increases in aprocess of receiving and processing signals, according to a vibrationinput and a user's voice, from individual sensors. Accordingly, atechnology for clearly and efficiently sensing various types of userinputs is required.

SUMMARY

Provided are an apparatus and method for signal processing. Provided arenon-transitory computer-readable recording media having recorded thereoncomputer programs, which, when executed by a computer, perform themethods.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of embodiments of the disclosure.

In accordance with an aspect of the disclosure, a signal processingapparatus includes a frequency detector configured to receive a userinput including at least one of a vibration input and a user voice;vibrate in response to the received user input; and detect a frequencyof the user input, based on the vibration; and a processor configured todetermine a type of the user input received by the frequency detector,based on the frequency detected by the frequency detector; and perform afunction corresponding to the user input of the determined type.

The frequency detector may include a plurality of vibration structuresthat sense vibration inputs and sounds of different frequency bands, andeach of the plurality of vibration structures may include a vibratorconfigured to vibrate based on the frequency of the user input as theuser input is received.

Each of the plurality of vibration structures may further include avibration detector configured to receive a vibration of the vibrator andgenerate an electrical signal corresponding to the received vibration.

The processor may be further configured to determine that the type ofthe received user input is the vibration input when the detectedfrequency is in a first frequency band; and determine that the type ofthe received user input is the user voice when the detected frequency isin a second frequency band.

The processor may be further configured to determine that the type ofthe received user input is the user voice when the received user inputlasts longer than a first length of time and the frequency of thereceived user input is in the second frequency band.

The processor may be further configured to determine that the type ofthe received user input is the vibration input when the frequency of thereceived user input is in an overlapping frequency band that is in boththe first frequency band and the second frequency band while thereceived user input lasts.

The processor may be further configured to determine that the type ofthe received user input is the user voice when the frequency of thereceived user input is in both the first frequency band and in a portionof the second frequency band not overlapping the first frequency band.

The first frequency band may correspond to a resonance frequency band ofa housing of the signal processing apparatus, and the second frequencyband may correspond to a voice grade.

The first frequency band may be 300 Hz to 500 Hz, and the secondfrequency band may be 300 Hz to 3400 Hz.

The vibration input may include at least one of a tap input, a swipeinput, and a bone conduction input.

The processor may be further configured to determine that the type ofthe received user input is the tap input when the frequency of the userinput is in the first frequency band while the received user inputlasts, and a duration of the user input is less than or equal to a firstlength of time, and determine that the type of the user input is theswipe input or the bone conduction input when the duration of the userinput exceeds the first length of time.

The processor may be further configured to determine that the type ofthe received user input is the tap input when the frequency of the userinput is in the first frequency band while the received user inputlasts, and a magnitude of an electrical signal generated by thefrequency detector, based on the vibration, is equal to or greater thana threshold level, and determine that the type of the user input is theswipe input or the bone conduction input when the magnitude of theelectrical signal is less than the threshold level.

The processor may be further configured to perform a first functioncorresponding to the user voice and a second function corresponding tothe vibration input when the received user input is determined toinclude both the user voice and the vibration input.

The processor may be further configured to identify a user who uses thesignal processing apparatus, based on a combination of the user voiceand the vibration input.

The signal processing apparatus may further include a display configuredto output visual information, and the processor may be furtherconfigured to control the display to display results of performing thefirst function corresponding to the user voice and the second functioncorresponding to the vibration input on different regions on thedisplay.

The signal processing apparatus may include an eyeglass wearable device,and the frequency detector may be arranged on an eyeglass frame of theeyeglass wearable device.

In accordance with an aspect of the disclosure, a signal processingmethod includes receiving a user input including at least one of avibration input and a user voice; vibrating in response to the receiveduser input; detecting a frequency of the received user input, based onthe vibration; detecting a type of the received user input, based on thedetected frequency; and performing a function corresponding to the userinput of the determined type.

The vibrating in response to the received user input may includevibrating based on the frequency of the received user input, wherein thevibrating is performed by a plurality of vibration structures forsensing vibration inputs and sounds of different frequency bands.

The detecting of the frequency of the received user input may furtherinclude generating an electrical signal corresponding to a vibration ofeach of the plurality of vibration structures.

The determining of the type of the received user input may includedetermining that the type of the received user input is the vibrationinput when the detected frequency is in a first frequency band, anddetermining that the type of the received user input is the user voicewhen the detected frequency is in a second frequency band.

The determining of the type of the received user input may furtherinclude determining that the type of the received user input is the uservoice when the received user input lasts longer than a first length oftime and the frequency of the lasting user input is in the secondfrequency band.

The determining of the type of the received user input may furtherinclude determining that the type of the received user input is thevibration input when the frequency of the received user input is in anoverlapping frequency band that is in both the first frequency band andthe second frequency band while the received user input lasts.

The determining of the type of the received user input may furtherinclude determining that the type of the received user input is the uservoice when the frequency of the received user input is in both the firstfrequency band and in a portion of the second frequency band notoverlapping the first frequency band.

The determining of the type of the received user input may furtherinclude determining that the type of the received user input is a tapinput when the frequency of the received user input is in the firstfrequency band while the received user input lasts, and a duration ofthe user input is less than or equal to a first length of time, anddetermining that the type of the received user input is a swipe input ora bone conduction input when the duration of the received user inputexceeds the first length of time.

The determining of the type of the received user input may furtherinclude determining that the type of the received user input is a tapinput when the frequency of the received user input is in the firstfrequency band while the received user input lasts, and a magnitude ofan electrical signal generated based on the vibration is equal to orgreater than a threshold level, and determining that the type of theuser input is a swipe input or a bone conduction input when themagnitude of the electrical signal is less than the threshold level.

The performing of the function corresponding to the user input of thedetermined type may include performing a first function corresponding tothe user voice and a second function corresponding to the vibrationinput when the received user input is determined to include both theuser voice and the vibration input.

The performing of the function corresponding to the user input of thedetermined type may include identifying a user, based on a combinationof the user voice and the vibration input.

The performing of the function corresponding to the user input of thedetermined type may include displaying results of performing a firstfunction corresponding to the user voice and a second functioncorresponding to the vibration input on different regions on a display.

A non-transitory computer-readable recording medium may have recordedthereon a computer program, which, when executed by a computer, performsthe method of an above-noted aspect of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram of a structure of a signal processingapparatus according to an embodiment;

FIG. 2 is a block diagram of a structure of a frequency detectoraccording to an embodiment;

FIG. 3 illustrates a structure of the frequency detector;

FIGS. 4A through 4C are cross-sectional views of a vibration structureof FIG. 3 ;

FIG. 5 is a view for explaining a sound sensing method using ambientmicrophones, according to a comparative example;

FIG. 6 is a view for explaining a directivity pattern of a frequencydetector according to an embodiment;

FIG. 7 illustrates a result of a measurement of a directivity pattern ofa frequency detector;

FIG. 8 is a view for explaining signal processing by a signal processingapparatus, according to an embodiment;

FIG. 9 is a graph showing a result of measuring respective directivitypatterns of both of a frequency detector according to an embodiment andan ambient microphone;

FIGS. 10A and 10B are views illustrating arrangements of a vibrator withrespect to an utterance point of a user voice;

FIG. 11 is a view illustrating a sound adjustment process by a soundadjuster, according to an embodiment;

FIG. 12 is a view illustrating a user voice signal generated by afrequency detector according to an embodiment;

FIG. 13 is a view for explaining an operation method of a signalprocessing apparatus, according to an embodiment;

FIG. 14 illustrates a frequency detector that receives a vibration inputand a user voice, according to an embodiment;

FIGS. 15A through 15C are views for explaining a method of determiningthe type of a user input;

FIG. 16 is a view for explaining a method of determining the type of auser input, based on the intensity of a signal, according to anembodiment;

FIGS. 17A and 17B are views illustrating results of sensing a vibrationinput;

FIGS. 18A and 18B are diagrams showing displays according toembodiments;

FIGS. 19A through 19C are views illustrating embodiments in which asignal processing apparatus is an eyeglasses-type wearable device;

FIG. 20 is a view illustrating an aspect in which a vibration input isreceived by a signal processing apparatus according to an embodiment;

FIGS. 21A through 21F are views illustrating operations of a signalprocessing apparatus with respect to various vibration inputs; and

FIG. 22 is a flowchart of a signal processing method according to anembodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings, wherein like referencenumerals refer to the like elements throughout. In this regard,embodiments may have different forms and should not be construed asbeing limited to the descriptions set forth herein. Accordingly,embodiments are merely described below, by referring to the figures, toexplain aspects of the present description. As used herein, the term“and/or” includes any and all combinations of one or more of theassociated listed items. Expressions such as “one or more of,” whenpreceding a list of elements, modify the entire list of elements and donot modify the individual elements of the list.

Although general terms widely used at present were selected fordescribing the disclosure in consideration of the functions thereof,these general terms may vary according to intentions of one of ordinaryskill in the art, case precedents, the advent of new technologies, orthe like. Terms arbitrarily selected by the applicant of the disclosuremay also be used in a specific case. In this case, their meanings needto be given in the detailed description. Hence, the terms must bedefined based on their meanings and the contents of the entirespecification, not by simply stating the terms.

Throughout the descriptions of embodiments, when an element is referredto as being “connected” or “coupled” to another element, it can bedirectly connected or coupled to the other element, or can beelectrically connected or coupled to the other element with interveningelements interposed therebetween. The terms “comprises” and/or“comprising” or “includes” and/or “including” when used in thisspecification, specify the presence of stated elements, but do notpreclude the presence or addition of one or more other elements.

Terms “configured” or “include” used herein should not be construed asnecessarily including all of several components or several steps writtenin the specification, but as not including some of the components orsteps or as further including additional components or steps.

While such terms as “first”, “second”, etc., may be used to describevarious components, such components must not be limited to the aboveterms. The above terms are used only to distinguish one component fromanother.

The descriptions of embodiments below should not be construed aslimiting the right scope of the accompanying claims, and it should beconstrued that all of the technical ideas included within the scopeequivalent to the claims are included within the right scope ofembodiments. Example embodiments of the disclosure will now be describedmore fully with reference to the accompanying drawings.

FIG. 1 is a block diagram of a structure of a signal processingapparatus according to an embodiment.

Referring to FIG. 1 , a signal processing apparatus 100 may include afrequency detector 110 and a processor 120. FIG. 1 illustrates onlycomponents of the signal processing apparatus 100 that are relevant toembodiments. Accordingly, it will be obvious to one of ordinary skill inthe art that the signal processing apparatus 100 may further includegeneral-use components other than those shown in FIG. 1 .

The signal processing apparatus 100 may be a wearable device worn by auser to receive a user input. Alternatively, the signal processingapparatus 100 may not be worn by a user, and may be arranged adjacent toa sound output apparatus or may be included in the sound outputapparatus. For example, when the signal processing apparatus 100 isincluded in a sound output apparatus, the user may perform functions bythe signal processing apparatus 100 by inputting a user input to thesound output apparatus.

However, this is merely an example, and the signal processing apparatus100 may be implemented by being modified in various forms capable ofreceiving a user input. Examples of the signal processing apparatus 100will be described later with reference to FIG. 19A.

The frequency detector 110 may receive a user input including at leastone of a vibration input and a user voice. The frequency detector 110may vibrate in response to the received user input. Vibration of thefrequency detector 110 will be described later with reference to FIG. 2.

The frequency detector 110 may detect frequency information of thereceived user input, based on the vibration according to the receiveduser input. For example, the frequency detector 110 may detectinformation including the amplitude, intensity, or the like according totime, for each frequency band of a user input received based on avibration of each vibrator.

A vibration input may be an input generated when a user using or wearingthe signal processing apparatus 100 contacts the signal processingapparatus 100, or may be an input transmitted when a vibration isgenerated on an already-contacted body part. For example, the vibrationinput may include a tap input, a swipe input, or a bone conductioninput. The bone conduction input refers to transmission of a vibrationthrough a skull, and may correspond to a vibration input generated via amovement of a face, air conduction within a skull, or the like when thesignal processing apparatus 100 is worn on a face. Various examples ofthe bone conduction input will be described later with reference toFIGS. 21C through 21F.

The frequency detector 110 may receive external sound generated outsidethe user, in addition to receiving the user input. For example, thefrequency detector 110 may receive overall sound including a user voiceand external sound. The user voice may correspond to the voice of theuser who uses or wears the signal processing apparatus 100. The externalsound is a sound received from the outside of the user, and maycorrespond to a sound excluding the user voice. For example, theexternal sound may include a voice of an outsider having a conversationwith the user, a sound output from an image viewed by the user, or asound generated in an environment around the user. The overall sound isa sound including both the user voice and the external sound, and maycorrespond to all sounds transmitted to (or received from) a signalprocessing apparatus. The overall sound may be transmitted to (orreceived from) the frequency detector 110, but the external sound may beattenuated from the overall sound by a structure or operation of thefrequency detector 110, and thus a user voice signal may be generated.

The frequency detector 110 may convert the received sound into anelectrical signal including information such as a frequency, amplitude,and time.

The frequency detector 110 may generate the user voice signal byattenuating the external sound from the received overall sound. Thefrequency detector 110 may generate a user voice signal in which a uservoice is made clearer, by attenuating the external sound. For example,the frequency detector 110 may attenuate a signal having directivity tothe user voice or corresponding to the external sound, based on athreshold value, in order to attenuate the received external sound. Thestructure and operation of the frequency detector 110 will be describedlater with reference to FIG. 2 .

The frequency detector 110 may receive sound through one surface formedby the frequency detector 110. The one surface may refer to a surfaceformed by a vibrator of the frequency detector 110 or may refer to asurface formed by a plurality of vibrators arranged in a two-dimensional(2D) manner. The frequency detector 110 may be arranged within thesignal processing apparatus 100 such that the surface formed by thefrequency detector 110 is arranged in a direction corresponding to atypical or expected utterance point of the user voice. Due to thisarrangement of the frequency detector 110, the user voice may be sensedwith a high sensitivity and the external sound may be sensed with a lowsensitivity. Accordingly, the external sound may be attenuated from theoverall sound received by the frequency detector 110, and the user voicesignal, which is a sound signal generated by the frequency detector 110,may be a signal in which the external sound has been attenuated.

For example, the frequency detector 110 may be arranged in the signalprocessing apparatus 100 such that an angle between one surfacereceiving the overall sound and a direction from the utterance point ofthe user voice to the one surface is about 60° to about 120°. Thearrangement of the frequency detector 110 (or the vibrator of thefrequency detector 110) will be described later with reference to FIGS.10A and 10B.

The processor 120 may determine a type of the user input received by thefrequency detector 110, based on the frequency information detected bythe frequency detector 110. Because the user input includes at least oneof the vibration input and the user voice, the processor 120 maydetermine whether the received user input is the vibration input or theuser voice or includes both the vibration input and the user voice. Amethod, performed by the processor 120, of determining the user inputwill be described later with reference to FIG. 13 and others.

The processor 120 may perform a function corresponding to a user inputof the determined type. For example, the processor 120 may perform afunction corresponding to a vibration input, a function corresponding toa user voice, or functions respectively corresponding to the two inputtypes. The processor 120 may perform a function corresponding to acombination of the vibration input and the user voice. The processor 120may identify the user, based on a combination of the vibration input andthe user voice. Because users have different voices and providedifferent movements of the skins in contact with the signal processingapparatus 100 and different bone conductions during utterance, theprocessor 120 may perform user authentication by identifying the user,based on the combination of the vibration input and the user voice.

The processor 120 may be implemented by an array of a plurality of logicgates, or by a combination of a general-use microprocessor and a memoryin which a program executable by the general-use microprocessor isstored. It will also be understood by one of ordinary skill in the artto which the embodiment pertains that the processor 120 may beimplemented by other types of hardware.

The signal processing apparatus 100 may sense both the user voice andthe vibration input by using one sensor called the frequency detector110, without using both of a special acoustic sensor and a specialvibration sensor in order to sense the user voice and the vibrationinput. Accordingly, the signal processing apparatus 100 does not requirecomparisons and calculations with respect to signals generated fromspecial sensors, thus enabling efficient processing. Moreover, thesignal processing apparatus 100 senses the user voice and the vibrationinput by using only the frequency detector 110, thus enabling reductionsin process costs and power consumption and achieving deviceminiaturization.

FIG. 2 is a block diagram of a structure of the frequency detector 110according to an embodiment.

Referring to FIG. 2 , the frequency detector 110 may include a pluralityof vibration structures 111. Each of the vibration structures 111 mayinclude a vibrator 112 and a vibration detector 113. Only componentsrelated with embodiments from among the components of the frequencydetector 110 are shown in FIG. 2 . Accordingly, it is obvious to one ofordinary skill in the art that the frequency detector 110 may furtherinclude general-use components other than those shown in FIG. 2 . Forexample, the frequency detector 110 may further include a support, asound adjuster, or the like.

The frequency detector 110 may include a plurality of vibrationstructures that sense vibration inputs and sounds of different frequencybands. The plurality of vibration structures may be formed in differentshapes (e.g., a length, a thickness, a shape, or a weight) and may haveresonance frequencies corresponding to the shapes. The plurality ofvibration structures may sense a vibration input and sound of afrequency band corresponding to each resonance frequency. A detailedstructure of the vibration structure 111 will be described later withreference to FIGS. 3 and 4A.

The vibrator 112 may vibrate in response to a user input. For example,the vibrator 112 may vibrate in response to a user input of a frequencyclose to a resonance frequency. The vibrator 112 may form one surfacethat receives a vibration input or sound. As vibrators are arranged in a2D manner within the frequency detector 110, the frequency detector 110may form one surface corresponding to a plurality of surfaces of thevibrators. The vibrator 112 may vibrate in a direction orthogonal to theone surface, in response to a vibration input or sound, based on thefrequency of the received vibration input or sound. The one surfaceformed by the vibrator 112 will be described later with reference toFIG. 4A.

The vibration detector 113 may receive a vibration of the vibrator 112and may generate an electrical signal corresponding to the receivedvibration. As the vibration is converted into the electrical signal bythe vibration detector 113, a signal processing apparatus may performvarious processing and calculations with respect to a received userinput.

FIG. 3 illustrates a structure of the frequency detector 110.

Referring to FIG. 3 , the frequency detector 110 may include a support115 and the plurality of vibration structures 111. The support 115 maybe formed to penetrate through a cavity 116. A silicon substrate, forexample, may be used as the support 115, but embodiments of thedisclosure are not limited thereto.

The plurality of vibration structures 111 may be arranged in a certainshape on the cavity 116 of the support 115. The vibration structures 111may be arranged on a plane without overlapping each other. Each of thevibration structures 111 may have one end fixed to the support 115 andthe other end extending toward the cavity 116, as illustrated in FIG. 3.

The vibration structures 111 may be provided to sense, for example,vibration input frequencies and acoustic frequencies of different bands.In other words, the vibration structures 111 may have different centerfrequencies or resonance frequencies. To this end, the vibrationstructures 111 may be provided to have different dimensions. Dimensionsof the vibration structures 111 may be set in consideration of desiredresonance frequencies for the vibration structures 111.

FIGS. 4A through 4C are cross-sectional views of the vibration structure111 of FIG. 3 .

Referring to FIG. 4A, the vibration structure 111 may include thevibrator 112, the vibration detector 113, and mass 114. The vibrationstructure 111 may have one end fixed to the support 115 and the otherend extending toward a cavity, as illustrated in FIGS. 4A through 4C.

Each vibration structure 111 may include the vibrator 112 vibrating inresponse to a received user input, and the vibration detector 113sensing a movement of the vibrator 112. The vibration structure 111 mayfurther include the mass 114 for providing a certain amount of mass tothe vibrator 112.

The vibrator 112 may vibrate based on the frequency of a receivedvibration input or sound. The vibrator 112 may greatly vibrate as thefrequency of the received vibration input or sound approaches aresonance frequency, and may have a small vibration as the frequency ofthe received vibration input or sound moves away from the resonancefrequency. Alternatively, the vibrator 112 may vibrate when a vibrationinput or sound of a sensible frequency band is received, and may notvibrate when a vibration input or sound deviating from the sensiblefrequency band is received.

Referring to FIGS. 4B and 4C, the vibrator 112 may form one surface 112a that receives a vibration input and sound.

The vibrator 112 may vibrate in a direction orthogonal to the onesurface 112 a, in response to the vibration input or sound. The vibrator112 may vibrate with an intensity based on an angle formed by apropagation direction 41 of the received sound and the one surface 112a. The vibrator 112 may vibrate with a large vibration intensity as theangle formed by the propagation direction 41 of the received sound andthe one surface 112 a approaches 90°, and may vibrate with a smallvibration intensity as the angle formed by the propagation direction 41of the received sound and the one surface 112 a approaches 0°.

As shown in FIG. 4B, in response to sound propagated at an angle of 90°with respect to the one surface 112 a, the vibrator 112 may vibrate witha largest vibration intensity. As shown in FIG. 4C, in response to soundpropagated at a smaller angle than 90° with respect to the one surface112 a, the vibrator 112 may vibrate with a smaller vibration intensitythan FIG. 4B.

Due to the vibration operation of the vibrator 112, a frequency detector(or vibration structures) may be arranged within a signal processingapparatus in consideration of the propagation direction 41 of the sound.For example, the frequency detector may be arranged in the signalprocessing apparatus so that a user voice is propagated to the onesurface 112 a at an angle close to 90°. In other words, the frequencydetector may be arranged such that the one surface 112 a is orientedtoward an utterance point of the user voice, and this arrangement willbe described later with reference to FIGS. 10A and 10B.

FIG. 5 is a view for explaining a sound sensing method using ambientmicrophones according to a comparative example.

The sound sensing method according to a comparative example of FIG. 5may use a plurality of ambient microphones 510 in order to maximizesound in a certain direction. The plurality of ambient microphones 510may be arranged at intervals of a certain distance D, a time or phasedelay in which sound reaches each of the ambient microphones 510 mayoccur due to the distance D, and overall directivity may be controlledby varying the degree of compensating for the time or phase delay. Thisdirectivity adjusting method may be referred to as a time difference ofarrival (TDOA).

However, because the above-described method is based on the premise thatthere is a difference in the time for sound to arrive at each ambientmicrophone, the interval needs to be set in consideration of thewavelength of an audible frequency band, and thus there may be alimitation in setting the interval between the ambient microphones 510.Because there is a limitation in setting the interval, there may be alimitation in miniaturization of an apparatus that performs theabove-described method. In particular, because a low frequency has along wavelength, the interval between the ambient microphones 510 mayneed to be wide, and a signal-to-noise ratio (SNR) of each ambientmicrophone may need to be high, in order to identify sound of a lowfrequency.

In addition, in the above-described method, because a phase variesaccording to the frequency band of sound sensed by each ambientmicrophone, the phase may need to be compensated for each frequency. Inorder to compensate for the phase for each frequency, theabove-described method may require a complex signal-processing processof applying an appropriate weight to each frequency.

Unlike the comparative example of FIG. 5 , a signal processing apparatusaccording to an embodiment does not have restrictions on the intervalbetween microphones, and may obtain a sound in a specific direction bydistinguishing directions only with a simple arithmetic operationwithout complex signal-processing. Efficient structure and operation ofthe signal processing apparatus will now be described in detail withreference to the drawings below.

FIG. 6 is a view for explaining a directivity pattern of a frequencydetector according to an embodiment.

Referring to FIG. 6 , the frequency detector 110 may have bidirectionaldirectivity patterns 61 and 62. For example, the bidirectionaldirectivity patterns 61 and 62 may be a figure-8 directivity patternincluding a front portion 61 oriented toward a front side (+z direction)of the frequency detector 110 and a rear portion 62 oriented toward arear side (−z direction) of the frequency detector 110.

When sound is perpendicularly propagated to the one surface 112 a formedby the vibrator 112, the vibrator 112 may vibrate with a large vibrationintensity by responding most sensitively. Accordingly, a directivitypattern based on a front direction (+z direction) and a rear direction(−z direction) of the frequency detector 110, which are perpendicular tothe one surface 112 a, may be formed. In this case, the frequencydetector 110 may respond with a reduced sensitivity with respect tosound received in a direction not oriented with respect to the frequencydetector 110 (for example, +x direction and −x direction). Accordingly,the frequency detector 110 may attenuate sound received in the directionnot oriented with respect to the frequency detector 110 (for example, +xdirection and −x direction).

A unidirectional pattern in the +z direction or the −z direction may beformed by blocking reception of sound by one surface according to thestructure of the frequency detector 110. The above-described directivitypatterns of the frequency detector 110 are merely examples, and adirectivity pattern may be variously modified according to arrangementsof vibration structures (or vibrators).

FIG. 7 illustrates a result of a measurement of a directivity pattern ofa frequency detector.

As shown in FIG. 7 , it may be seen that the frequency detector has auniform bi-directional directivity pattern for various frequencies. Inother words, it may be seen that the frequency detector hasdirectionality in the +z-axis direction and the −z-axis direction ofFIG. 3 , which are a 0° direction and a 180° direction, for variousfrequencies.

FIG. 8 is a view for explaining signal processing by a signal processingapparatus according to an embodiment.

Referring to FIG. 8 , the frequency detector 110 may have abidirectional directivity pattern 81, and an ambient microphone 510 mayhave an omnidirectional or non-directional directivity pattern 82. Thefrequency detector 110 may sense sound in phase with the sound sensed bythe ambient microphone 510, from a front direction (for example, the +zdirection of FIG. 5 ) of the bidirectional directivity pattern 81, andmay sense sound in anti-phase with the sound sensed by the ambientmicrophone 510, from a rear direction (for example, the −z direction ofFIG. 5 ) of the bidirectional directivity pattern 81. However, thedirectivity pattern of the frequency detector 110 shown in FIG. 8 ismerely an example. As described above, the directivity pattern may bevariously modified according to structures of the frequency detector 110and arrangements of vibration structures (or vibrators).

FIG. 9 is a graph showing a result of measuring respective directivitypatterns of a frequency detector according to an embodiment and anambient microphone.

Referring to FIG. 9 , the frequency detector has a bidirectionaldirectivity pattern, and the ambient microphone has an omni-directional(or non-directional) directivity pattern. For example, the frequencydetector may sense sound transmitted by a 330° to 30° (60° to 120° withrespect to one surface formed by the frequency detector) regioncorresponding to a front side (+z direction of FIG. 6 ), and may sensesound transmitted by a 150° to 210° (240° to 300° with respect to theone surface formed by the frequency detector) region corresponding to arear side (−z direction of FIG. 6 ). For example, the frequency detectormay sense a sound having a magnitude of approximately 0.85 times aslarge in a 30° (120° based on the one surface formed by the frequencydetector) region as that in a 0° (90° based on the one surface formed bythe frequency detector) region.

The ambient microphone may sense sound transmitted from all directionsof an ambient 360° region.

The frequency detector may attenuate sound received in a direction closeto 90° or 270° (0° based on the one surface formed by the frequencydetector). According to the embodiment of FIG. 9 , the frequencydetector may react with a low sensitivity with respect to sound receivedin a direction of 60° to 120°, and thus may attenuate the sound in thedirection of 60° to 120°.

FIG. 9 illustrates only a result of one frequency. However, as describedabove with reference to FIG. 7 , because the frequency detector may havea uniform sensitivity with respect to various frequencies, results forvarious frequencies may form directivity patterns having similar shapes.For example, various frequencies may be the frequencies of an audiblefrequency region, and a directivity pattern having a similar shape maybe formed for the frequency detector, regardless of the frequency.

FIGS. 10A and 10B are views illustrating arrangements of a vibrator withrespect to an utterance point of a user voice.

Referring to FIGS. 10A and 10B, a user voice propagated from anutterance point 42 of the user voice may be received by the one surface112 a formed by the vibrator 112.

As shown in FIG. 10A, when a propagation direction of the user voice andthe one surface 112 a formed by the vibrator 112 are perpendicular toeach other, the vibrator 112 may respond with a highest sensitivity, andthe user voice may be sensed loudest. Thus, the frequency detector maybe arranged within a signal processing apparatus such that the onesurface 112 a formed by the vibrator 112 (or a plurality of vibrators)is arranged in a direction corresponding to the utterance point 42 ofthe user voice.

In other words, the frequency detector may be arranged such that the onesurface 112 a formed by the vibrator 112 (or a plurality of vibrators)and a direction from the utterance point 42 of the user voice to the onesurface 112 a correspond to each other (preferably, form an angle of90°).

When the one surface 112 a and the propagation direction of the uservoice form an angle of 90°, sound may be sensed with a highestsensitivity, but several restrictions in process or use may make itdifficult to keep the angle at 90°. For example, as shown in FIG. 10B,the propagation direction of the user voice and the one surface 112 amay form an angle of less than 90°. However, also in this case, thefrequency detector may sense the user voice, as described above withreference to FIG. 9 .

The frequency detector may be arranged within the signal processingapparatus at an angle for securing flexibility in process and use andeffectively sensing the user voice. The frequency detector may bearranged within the signal processing apparatus such that the onesurface 112 a formed by the vibrator 112 (or a plurality of vibrators)and the direction from the utterance point 42 of the user voice to theone surface 112 a form an angle of 60° to 120°. As described above withreference to FIG. 9 , even when the frequency detector receives sound at60° or 120°, the frequency detector may receive the sound with amagnitude of about 0.85 times as that when the frequency detectorreceives sound at 90°. Thus, 60° to 120° may be an angle sufficient toprovide flexibility in process and use and sense the user voice.

When the frequency detector is arranged to be oriented toward theutterance point 42 of the user voice, the frequency detector mayrespond, with a low sensitivity, to external sound generated in a placespaced apart from the utterance point 42 of the user voice. Thus, thefrequency detector may attenuate the external sound.

An embodiment in which this arrangement of the frequency detector isapplied to a signal processing apparatus will be described later withreference to FIG. 19C.

FIG. 11 is a view illustrating a sound adjustment process by a soundadjuster according to an embodiment.

Referring to FIG. 11 , electrical acoustic signal frames 1110 a through1110 f generated by three vibration structures that sense differentfrequency bands are shown in two time frames. Acoustic signal frames maybe input to a sound adjuster 1100, and one sound adjuster 1100 may beincluded in each vibration structure or may be included in a frequencydetector.

The sound adjuster 1100 of the frequency detector may determine anelectrical signal that is to be attenuated, from among electricalsignals generated by vibration structures, based on a threshold value.The sound adjuster 1100 may attenuate the determined electrical signal.The electrical signal that is attenuated may be a signal correspondingto external sound. As the signal corresponding to external sound isattenuated by the sound adjuster 1100, a user voice may be maximized.

“Frame 0” indicates an acoustic signal frame measured during a firsttime section. “Frame j” indicates an acoustic signal frame measuredduring a j-th time section, after the first time section. First throughthird acoustic signal frames 1110 a through 1110 c are frames measuredin the same time section (first time section), and fourth through sixthacoustic signal frames 1110 d through 1110 f are frames measured in thesame time section (j-th time section).

The first and fourth acoustic signal frames 1110 a and 1110 d may be inthe same frequency band, and may be input to the sound adjuster 1100through the same vibration structure. The second and fifth acousticsignal frames 1110 b and 1110 e may be in the same frequency band, andmay be input to the sound adjuster 1100 through the same vibrationstructure. The third and sixth acoustic signal frames 1110 c and 1110 fmay be in the same frequency band, and may be input to the soundadjuster 1100 through the same vibration structure. The frequency bandof the first and fourth acoustic signal frames 1110 a and 1110 d, thefrequency band of the second and fifth acoustic signal frames 1110 b and1110 e, and the frequency band of the third and sixth acoustic signalframes 1110 c and 1110 f are different from one another.

In FIG. 11 , “Drop” indicates a case where the sound adjuster 1100determines an input acoustic signal as an acoustic signal to beattenuated, and “Add” indicates a case where the sound adjuster 1100does not attenuate the input acoustic signal.

Referring to FIG. 11 , when the intensity of an acoustic signal is equalto or less than a threshold value T or exceeds the threshold value T bya degree equal to or less than a set value as in the case of the firstthrough fourth acoustic signal frames 1110 a through 1110 d, the soundadjuster 1100 may attenuate the acoustic signal (Drop).

On the other hand, when the intensity of an acoustic signal exceeds thethreshold value T and the degree of excess is higher than a preset valueas in the case of the fifth and sixth acoustic signal frames 1110 e and1110 f, the sound adjuster 1100 may not attenuate the acoustic signal(Add).

An output result of the sound adjuster 1100 may be transmitted to theprocessor 120 via, for example, an amplifier.

FIG. 12 is a view illustrating a user voice signal generated by afrequency detector according to an embodiment.

FIG. 12 illustrates a first graph 1210 showing a result of sensing auser voice by using the method according to the comparative example ofFIG. 5 , and a second graph 1220 showing a result of sensing of a uservoice by the frequency detector.

The first graph 1210 shows a result of attenuating external sound byusing a plurality of ambient microphones according to the comparativeexample of FIG. 5 . The first graph 1210 shows a signal 1210 acorresponding to a user voice, and a signal 1210 b corresponding toexternal sound. It is confirmed that the signal 1210 b corresponding toexternal sound was attenuated more than the signal 1210 a correspondingto a user voice but remains at a detectable level.

The second graph 1220 shows a user voice signal generated by thefrequency detector attenuating an external acoustic signal. The secondgraph 1220 shows a signal 1220 a corresponding to a user voice, and asignal 1220 b corresponding to external sound. It is seen in the secondgraph 1220 that the signal 1220 b corresponding to external sound wasclearly attenuated. It is seen in the second graph 1220 that the signal1220 b corresponding to external sound was attenuated to a level closeto a silence as an undetectable level.

The frequency detector may attenuate external sound through adirectivity-based arrangement of vibration structures toward anutterance point of a user voice. Alternatively, the frequency detectormay attenuate external sound by attenuating some signals from amongsignals generated by the vibration structures, based on a thresholdvalue. As a result, the frequency detector may attenuate an externalacoustic signal by using one or both of the two above-described methodsand may generate the user voice signal.

FIG. 13 is a view for explaining an operation method of a signalprocessing apparatus, according to an embodiment.

Referring to FIG. 13 , a user input may be received by the frequencydetector 110, and frequency information generated by the frequencydetector 110 may be input to the processor 120. The processor 120 maydetermine a type of the user input, based on the received frequencyinformation.

The frequency detector 110 may vibrate based on the frequency of theuser input, and may generate an electrical signal corresponding to theuser input by sensing the vibration. The electrical signal may includethe frequency information of the user input, and the frequencyinformation may include information about the intensity or amplitude ofeach frequency of the user input according to time.

The processor 120 may determine a type of the user input, based on thefrequency band of the user input, a duration of the user input, themagnitude of a signal, and the like. A method, performed by theprocessor 120, of determining the type of the user input will bedescribed later with reference to FIGS. 14 through 16 .

FIG. 14 illustrates a frequency detector that receives a vibration inputand a user voice, according to an embodiment.

The frequency detector 110 may detect frequency information of avibration input and frequency information of a user voice. In otherwords, the signal processing apparatus 100 may sense both the vibrationinput and the user voice by using only the frequency detector 110without using special sensors for sensing the vibration input and theuser voice.

Corresponding vibration structures may vibrate according to thefrequencies of the user input received by the frequency detector 110.When high-frequency sound and low-frequency sound are both received,vibration structures having a resonance frequency of a high frequencyand vibration structures having a resonance frequency of a low frequencymay vibrate. When the vibration input is received, vibration structureshaving a resonance frequency corresponding to the vibration input mayvibrate. For example, because a housing 130 of the signal processingapparatus 100 vibrates in response to the vibration input, vibrationstructures having a resonance frequency of the housing 130 may vibrate.

The frequency detector 110 may detect a frequency band, a time period(e.g., a length of time), and an intensity for vibrations of thevibration structures by sensing the vibrations of the vibrationstructures, and may generate an electrical signal including frequencyinformation, based on the detected frequency band, the detected timeperiod, and the detected intensity.

FIGS. 15A through 15C are views for explaining a method of determiningthe type of a user input.

FIGS. 15A through 15C illustrate graphs showing a signal includingfrequency information detected from the user input by a frequencydetector. The graphs show a first frequency band 151 and a secondfrequency band 152, and also show first through seventh user inputs 153a through 153 g.

A processor may determine a frequency band including a frequencydetected by the frequency detector from among the first frequency band151 and the second frequency band 152. The first frequency band 151 maycorrespond to the resonance frequency of a housing of a signalprocessing apparatus, and the second frequency band 152 may correspondto a voice grade. Because a housing of an electronic device is generallyformed of a material such as plastic or iron, the resonance frequency ofthe housing may belong to a low frequency band compared to the voicegrade. Accordingly, the processor may determine a signal of the firstfrequency band 151, which is a relatively low frequency band, as avibration input, and may determine a signal of the second frequency band152, which is a relatively high frequency band, as a user voice.

When the frequency of the first frequency band 151 is detected, theprocessor may determine the type of the user input as a vibration input.For example, while a signal corresponding to a user input is beingmaintained, when a detected frequency is included in the first frequencyband 151 without deviating from the first frequency band 151, theprocessor may determine the type of the user input as a vibration input.When the frequency of the second frequency band 152 is detected, theprocessor may determine the type of the user input as a user voice.

According to the embodiment of FIG. 15A, the first frequency band 151and the second frequency band 152 may be different from each otherwithout a common frequency band. For example, the first frequency band151 may be about 300 Hz to about 500 Hz, and the second frequency band152 may be about 500 Hz to about 3400 Hz. The processor may determine afirst user input 153 a having the frequency of the first frequency band151 as a vibration input, and may determine a second user input 153 bhaving the frequency of the second frequency band 152 as a user voice.

The first frequency band 151 and the second frequency band 152 may havea certain range in common (e.g., an overlapping frequency band that isin both the first frequency band 151 and the second frequency band 152).While a user input is being maintained, when the frequency of the userinput is included in a common frequency band of the first frequency band151 and the second frequency band 152, the processor may determine atype of the user input as a vibration input.

When a user input lasts longer than a first time t1 and the frequency ofthe user input is included in the second frequency band 152 during theuser input, the processor may determine a type of the user input as auser voice. When a signal lasting for the duration of or shorter thanthe first time t1 in the second frequency band 152 is detected, theprocessor may determine the detected signal as noise rather than a userinput.

When the frequency of a user input is included in both the firstfrequency band 151 and a frequency band excluding the common frequencyband from the second frequency band 152 (e.g., a portion of the secondfrequency band not overlapping the first frequency band), the processormay determine the user input as a user voice.

According to the embodiment of FIG. 15B, the first frequency band 151and the second frequency band 152 may have a certain portion in common.For example, the first frequency band 151 may be about 300 Hz to about500 Hz, and the second frequency band 152 may be about 300 Hz to about3400 Hz. The processor may determine, as a vibration input, a third userinput 153 c of which a frequency is included in the common frequencyband of the first frequency band 151 and the second frequency band 152.Because the third user input 153 c lasts during or shorter than thefirst time t1, the processor may determine the third user input 153 c asa tap input from among vibration inputs. The processor may determine, asa user voice, a fourth user input 153 d of which a frequency is notincluded in the common frequency band and is included in only the secondfrequency band 152 and lasts longer than the first time t1. Theprocessor may determine, as a user voice, a fifth user input 153 e ofwhich a frequency is included in both the first frequency band 151 andthe frequency band excluding the common frequency band from the secondfrequency band 152.

When the processor determines the type of the user input as a vibrationinput, the processor may also determine a type of the vibration input.For a signal of the first frequency band 151, when the duration of auser input is less than or equal to the first time t1, the processor maydetermine a type of the user input as a tap input. For a signal of thefirst frequency band 151, when the duration of the user input is greaterthan the first time t1, the processor may determine a type of the userinput as a swipe input or a bone conduction input. The first time t1 isa time length for distinguishing a tap input from the other vibrationinputs, and may correspond to, for example, about 0.3 seconds to about0.6 seconds.

According to the embodiment of FIG. 15C, the processor may determine, asa tap input, a sixth user input 153 f having a frequency included in thefirst frequency band 151 during an input and lasting for the duration ofor shorter than the first time t1. The processor may determine, as aswipe input or a bone conduction input, a seventh user input 153 ghaving a frequency included in the first frequency band 151 during aninput and lasting longer than the first time t1.

FIG. 16 is a view for explaining a method of determining the type of auser input, based on the intensity of a signal, according to anembodiment.

FIG. 16 illustrates a signal 163 a corresponding to an eighth user inputand a signal 163 b corresponding to a ninth user input, which aredetected from a first frequency band.

When the frequency of a received user input is included in the firstfrequency band, the processor may determine the user input as avibration input. The processor may determine the user input as avibration input and may also determine a type of the vibration input,based on the magnitude of the signal. The frequency detector may vibratein response to the user input, and may generate an electrical signalincluding frequency information, based on the vibration. The processormay determine the type of the vibration input by comparing the magnitudeof the generated electrical signal with a threshold level TL.

When a signal of the user input is equal to or greater than thethreshold level TL, the processor may determine the type of the userinput as a tap input. The tap input is an input of briefly tapping thehousing of the signal processing apparatus, and has a higher vibrationintensity than a swipe input and a bone conduction input that arerubbing or shaking inputs, and thus a large signal may be generated.

When the signal of the user input is less than the threshold level TL,the processor may determine the type of the user input as a swipe inputor a bone conduction input. The swipe input and the bone conductioninput have smaller vibration intensities than a tap input that is atapping input, and thus a small signal may be generated.

When the magnitude of the signal exceeds the threshold level TL and thedegree of excess exceeds a preset value, the processor may determine theuser input as a tap input. When the magnitude of the signal is less thanor equal to the threshold level TL or the degree of excess is less thanor equal to the preset value even when the magnitude of the signalexceeds the threshold level TL, the processor may determine the userinput as a swipe input or a bone conduction input.

According to the embodiment of FIG. 16 , because the signal 163 a of theeighth user input exceeds the threshold level TL, the processor maydetermine the eighth user input as a tap input. Because the signal 163 bof the ninth user input is less than the threshold level TL, theprocessor may determine the ninth user input as a swipe input or a boneconduction input.

FIGS. 17A and 17B are views illustrating results of sensing a vibrationinput.

FIG. 17A illustrates graphs showing a result of sensing a vibrationinput by using the ambient microphones 510 of FIG. 5 . A third graph1710 a shows the magnitude of a signal detected over time in the firstfrequency band, and a fourth graph 1720 a shows a spectrogram includingfrequency information over time. The third graph 1710 a and the fourthgraph 1720 a correspond to each other, and show signals received at anidentical time.

In the third graph 1710 a, four points where vibration inputs arereceived are marked. However, the third graph 1710 a shows that signalshaving greater power than vibration inputs are detected even at pointswhere the vibration inputs are not received, and that signals having toosmall power to be detected are detected even at the points where thevibration inputs are received. Thus, unless the times at which vibrationinputs are received are known in advance through the third graph 1710 a,the vibration inputs are not clearly distinguished.

The fourth graph 1720 a shows a result of reception of the vibrationinputs at the same points as those in the third graph 1710 a. However,in the fourth graph 1720 a, the vibration inputs are not clearlydistinguished in the first frequency band, which is the frequency bandof the vibration inputs.

Thus, when the ambient microphones 510 of FIG. 5 are used, the vibrationinputs are not sensed.

FIG. 17B illustrates graphs showing a result of sensing of vibrationinputs by a signal processing apparatus. A fifth graph 1710 b shows themagnitude of a signal detected over time in the first frequency band,and a sixth graph 1720 b shows a spectrogram including frequencyinformation over time. The fifth graph 1710 b and the sixth graph 1720 bcorrespond to each other, and show signals received at an identicaltime.

In the fifth graph 1710 b, five points where vibration inputs arereceived are marked. The fifth graph 1710 b shows that signals are notdetected at points where no vibration inputs are received, and thatsignals having power large enough to be detected are detected at thepoints where the vibration inputs are received. Thus, the times at whichthe vibration inputs are received are clearly distinguished through thefifth graph 1710 b, and thus the signal processing apparatus may sensethe vibration inputs when the vibration inputs are received.

The sixth graph 1720 b shows a result of reception of vibration inputsat the same points as those in the fifth graph 1710 b. The sixth graph1720 b shows that frequencies are clearly detected at times whenvibration inputs are received in the first frequency band, which is thefrequency band of the vibration inputs.

Thus, the signal processing apparatus may effectively sense a vibrationinput in any environment by clearly distinguishing the vibration inputfrom overall sound.

FIGS. 18A and 18B are diagrams showing displays according toembodiments.

The signal processing apparatus may further include a display 1800 thatoutputs visual information. The display 1800 may display a variety ofvisual information in response to control by the processor. Theprocessor may perform a function corresponding to a user voice (e.g., afirst function) or a function corresponding to a vibration input (e.g.,a second function). The processor may display a result of execution ofthe function on the display 1800. When the processor performs both thefunction corresponding to a user voice and the function corresponding toa vibration input, the processor may display results of execution of thefunctions on different regions of the display 1800.

Referring to FIG. 18A, the display 1800 may include a first region 1800a and a second region 1800 b within one frame. For example, the display1800 may display the result of execution of the function correspondingto a user voice on the first region 1800 a, and may display the resultof execution of the function corresponding to a vibration input on thesecond region 1800 b.

Referring to FIG. 18B, the display 1800 may include a first region 1800a and a second region 1800 b formed on independent frames. For example,the display 1800 may display the result of execution of the functioncorresponding to a user voice on the first region 1800 a, and maydisplay the result of execution of the function corresponding to avibration input on the second region 1800 b.

FIGS. 19A through 19C are views illustrating embodiments in which asignal processing apparatus is an eyeglass wearable device.

Referring to FIG. 19A, the signal processing apparatus 100 is aneyeglass wearable device and may include an eyeglass frame 1900. Theeyeglass frame 1900 may include an eyeglass bridge 1900 a, eyeglass rims1900 b, and eyeglass temples 1900 c.

A frequency detector may be arranged on the eyeglass frame 1900. Thefrequency detector may be arranged at various locations on the eyeglassframe 1900 according to inputs to be received. For example, thefrequency detector may be arranged on the eyeglass bridge 1900 a or theeyeglass rims 1900 b so as to receive a user voice at closer locations.The frequency detector may be arranged on the eyeglass temples 1900 c soas to be easily contacted by a hand of a user.

Although FIG. 19A illustrates that the signal processing apparatus 100is an eyeglass wearable device, this is merely an example. The signalprocessing apparatus 100 may be in the form of a watch or bracelet wornon the wrist, in the form of a necklace worn on the neck, or in variousother types of wearable devices such as earphones and headphones worn onthe ears. The signal processing apparatus 100 may correspond to any typeof wearable device as long as it may be worn by a user.

Referring to FIG. 19B, the frequency detector 110 may be arranged on theeyeglass bridge 1900 a of the signal processing apparatus 100.

Because an utterance point of a user voice corresponds to a mouth orlips of the user, the frequency detector 110 may be arranged on theeyeglass bridge 1900 a such as to correspond to the utterance point.Alternatively, the frequency detector 110 may be arranged on theeyeglass temples 1900 c to more effectively receive a vibration inputfrom the user's lateral direction. However, as described above, thefrequency detector 110 may be arranged at various locations within theeyeglass frame 1900.

Referring to FIG. 19C, a user voice is propagated from the utterancepoint 42 of the user voice toward the frequency detector 110.

The utterance point 42 of the user voice may be a location correspondingto the mouth or lips of the user. The user voice may be propagated tothe frequency detector 110 and may be received by the one surface 112 aof the vibrator 112 of the frequency detector 110. When the user voiceis propagated perpendicular to the one surface 112 a of the vibrator112, the user voice may be sensed with a highest sensitivity by thefrequency detector 110.

Thus, as shown in FIG. 19C, the frequency detector 110 may be arrangedin the signal processing apparatus 100 such that a direction from theutterance point 42 of the user voice to the one surface 112 a isperpendicular. When a voice of an outside is received from the frontside or lateral side of the user, the outside voice is received in adirection parallel to the one surface 112 a of the frequency detector110, and thus the outsider voice may be sensed with a lowest sensitivityby the frequency detector 110 or may not be sensed. Due to thisarrangement, the frequency detector 110 may attenuate external sound andemphasize the user voice. The user voice being emphasized may not referto the user voice being amplified, but refer to the user voice beingclarified due to attenuation of other signals.

However, because maintaining a perpendicular arrangement is difficultdue to restrictions in process or use, the frequency detector 110 may bearranged such that the one surface 112 a and a traveling direction ofthe user voice form 60° to 120°. As described above with reference toFIGS. 10A and 10B, even when the frequency detector 110 is disposed atthe above angle, the frequency detector 110 may effectively sense theuser voice from which the external sound has been attenuated.

FIG. 20 is a view illustrating an aspect in which a vibration input isreceived by a signal processing apparatus according to an embodiment.

Referring to FIG. 20 , a user inputs a vibration input to variouslocations on an eyeglass frame.

The vibration input may be received by various locations on the signalprocessing apparatus 100, such as the eyeglass bridge 1900 a, theeyeglass rims 1900 b, or the eyeglass temples 1900 c. A housing of thesignal processing apparatus 100 may vibrate in response to the vibrationinput, and the vibration may be transmitted to the frequency detector110, and thus the frequency detector 110 may also vibrate. The frequencydetector 110 may detect frequency information of the vibration input bysensing the vibration.

FIGS. 21A through 21F are views illustrating operations of the signalprocessing apparatus 100 with respect to various vibration inputs.

Referring to FIGS. 21A through 21F, the signal processing apparatus 100may receive various vibration inputs. A display may include a firstregion (for example, a right eyeglass based on a user) and a secondregion (for example, a left eyeglass based on the user) and may displaydifferent pieces of visual information on the first and second regions.The signal processing apparatus 100 may display visual informationcorresponding to a vibration input on the display. For example, thesignal processing apparatus 100 may display visual informationcorresponding to a user voice on the first region, and may displayvisual information corresponding to a vibration input on the secondregion.

The signal processing apparatus 100 may receive a vibration input, auser voice, and external sound generated in front of the signalprocessing apparatus 100. Because the external sound is received in thefront direction of the frequency detector 110, the external sound may bereceived parallel to one surface formed by a vibrator or at an angleclose to parallel to the one surface. Thus, the frequency detector 110may attenuate the external sound and may effectively sense the uservoice and the vibration input.

According to the embodiment of FIG. 21A, the signal processing apparatus100 may receive a tap input. The signal processing apparatus 100 mayplay or stop an image or music in response to the tap input, and maydisplay visual information corresponding to the play or stop on thedisplay.

According to the embodiment of FIG. 21B, the signal processing apparatus100 may receive a swipe input. The swipe input may be an input by a handof a user contacting the eyeglass temples 1900 c and then moving in onedirection until the contact is released. Due to the swipe input, avibration may occur in the eyeglass temples 1900 c and may betransmitted to the frequency detector 110. The signal processingapparatus 100 may transmit a message in response to the swipe input andmay display visual information corresponding to the message transmissionon the display.

For example, the processor may determine a vibration input lastinglonger than a first length of time and shorter than a second length oftime to be a swipe input. The second length of time may correspond to,for example, 1.5 seconds to 3 seconds.

According to the embodiments of FIGS. 21C through 21F, the signalprocessing apparatus 100 may receive a bone conduction input. Accordingto the embodiment of FIG. 21C, the signal processing apparatus 100 mayreceive a bone conduction input based on a vibration generated due touser's wind blowing. The signal processing apparatus 100 may transmit amessage in response to the bone conduction input and may display visualinformation corresponding to the message transmission on the display.

For example, the processor may determine a vibration input lastinglonger than the first length of time and shorter than the second lengthof time to be a bone conduction input generated due to user's windblowing. However, the processor may determine a user input as a swipeinput when a signal having a magnitude greater than or equal to a presetvalue is detected, and may determine a user input as a bone conductioninput generated due to user's wind blowing when a signal having amagnitude less than the preset value is detected.

According to the embodiment of FIG. 21D, the signal processing apparatus100 may receive a bone conduction input based on a vibration generateddue to user's breathing. The signal processing apparatus 100 maydetermine a breathing state of the user in response to the boneconduction input and may display visual information corresponding to thebreathing state on the display.

For example, the processor may determine a vibration input lastinglonger than the second length of time to be a bone conduction inputgenerated due to user's breathing.

According to the embodiment of FIG. 21E, the signal processing apparatus100 may receive a bone conduction input based on a vibration generateddue to user's coughing. The signal processing apparatus 100 maydetermine a health state of the user in response to the bone conductioninput and may display visual information corresponding to the healthstate on the display.

For example, when a signal lasting equal to or shorter than the firstlength of time and having a magnitude equal to or greater than thepreset value is repeatedly continued, the processor may determine a userinput as a bone conduction input caused by user's coughing.

According to the embodiment of FIG. 21F, the signal processing apparatus100 may receive a bone conduction input based on a vibration generateddue to user's teeth grinding. The signal processing apparatus 100 maydetermine a health state of the user in response to the bone conductioninput and may display visual information corresponding to the healthstate on the display.

For example, the processor may determine a user input as a boneconduction input caused by user's teeth grinding, based on the patternor the like of a repeated vibration input.

Various functions performed based on the above-described user inputs aremerely examples and may be variously modified and implemented.

FIG. 22 is a flowchart of a signal processing method according to anembodiment.

Referring to FIG. 22 , the signal processing method includes operationsserially performed by the signal processing apparatus 100 of FIG. 1 .Thus, although omitted, the description of the signal processingapparatus 100 given above with reference to FIG. 1 and the like may alsoapply to the signal processing method of FIG. 22 .

In operation 2210, the signal processing apparatus 100 may receive auser input including at least one of a vibration input and a user voice.

In operation 2220, the signal processing apparatus 100 may vibrate inresponse to the user input.

In the signal processing apparatus 100, a plurality of vibrationstructures that sense vibration inputs and sounds of different frequencybands may vibrate based on the frequency of the user input.

The signal processing apparatus 100 may generate electrical signalsrespectively corresponding to the vibrations of the plurality ofvibration structures.

In operation 2230, the signal processing apparatus 100 may detect thefrequency of the received user input, based on the vibrations.

The signal processing apparatus 100 may generate electrical signalsrespectively corresponding to the vibrations of the plurality ofvibration structures.

In operation 2240, the signal processing apparatus 100 may determine atype of the received user input, based on the detected frequency.

The signal processing apparatus 100 may determine the type of thereceived user input as a vibration input when a frequency of a firstfrequency band is detected, and may determine the type of the receiveduser input as a user voice when a frequency of a second frequency bandis detected.

When the received user input lasts longer than a first length of timeand the frequency of the lasting user input is included in the secondfrequency band, the signal processing apparatus 100 may determine thetype of the received user input as a user voice.

When the frequency of the received user input is included in a commonfrequency band of the first frequency band and the second frequency bandwhile the received user input is being continued, the signal processingapparatus 100 may determine the type of the received user input as avibration input.

When the frequency of the received user input is included in both thefirst frequency band and a frequency band excluding the first frequencyband from the second frequency band, the signal processing apparatus 100may determine the type of the received user input as a user voice.

When the frequency of the received user input is included in the firstfrequency band while the received user input is being continued, and theduration of the received user input is less than or equal to the firstlength of time, the signal processing apparatus 100 may determine thetype of the received user input as a tap input. On the other hand, whenthe duration of the received user input exceeds the first length oftime, the signal processing apparatus 100 may determine the type of thereceived user input as a swipe input or a bone conduction input.

When the frequency of the received user input is included in the firstfrequency band while the received user input is being continued, and themagnitude of an electrical signal generated based on a vibration isequal to or greater than a threshold level, the signal processingapparatus 100 may determine the type of the received user input as a tapinput. On the other hand, when the magnitude of the electrical signalgenerated based on the vibration is less than the threshold level, thesignal processing apparatus 100 may determine the type of the receiveduser input as a swipe input or a bone conduction input.

In operation 2250, the signal processing apparatus 100 may perform afunction corresponding to a user input of the determined type.

When the received user input is determined to include both a user voiceand a vibration input, the signal processing apparatus 100 may perform afunction corresponding to the user voice and a function corresponding tothe vibration input.

The signal processing apparatus 100 may identify the user, based on acombination of the user voice and the vibration input.

The signal processing apparatus 100 may display results of eachexecution of the function corresponding to the user voice and thefunction corresponding to the vibration input on different regions onthe display.

The signal processing apparatus 100 may sense both the user voice andthe vibration input by using one sensor called the frequency detector110, without using a special acoustic sensor and a special vibrationsensor in order to sense the user voice and the vibration input.Accordingly, the signal processing apparatus 100 does not requirecomparisons and calculations with respect to signals generated fromspecial sensors, thus enabling efficient processing. Moreover, thesignal processing apparatus 100 senses the user voice and the vibrationinput by using only the frequency detector 110, thus enabling reductionsin process costs and power consumption and achieving deviceminiaturization.

A computer readable storage medium may have embodied thereon at leastone program including instructions for performing the above-describedoperation method of FIG. 22 . Examples of the computer-readablerecording medium include a magnetic medium such as a hard disk, a floppydisk, or a magnetic tape, an optical medium such as a compactdisk-read-only memory (CD-ROM) or a digital versatile disk (DVD), amagneto-optical medium such as a floptical disk, and a hardware devicespecially configured to store and execute program commands such as aROM, a random-access memory (RAM), or a flash memory. Examples of theprogram commands are advanced language codes that can be executed by acomputer by using an interpreter or the like as well as machine languagecodes made by a compiler.

It should be understood that embodiments described herein should beconsidered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each embodimentshould typically be considered as available for other similar featuresor aspects in other embodiments. While one or more embodiments have beendescribed with reference to the figures, it will be understood by thoseof ordinary skill in the art that various changes in form and detailsmay be made therein without departing from the spirit and scope asdefined by the following claims.

What is claimed is:
 1. A signal processing apparatus comprising: afrequency detector configured to: receive a user input comprising atleast one of a vibration input and a user voice; vibrate in response tothe received user input; and detect a frequency of the user input, basedon the vibration; and a processor configured to: determine a type of theuser input received by the frequency detector, based on the frequencydetected by the frequency detector; and perform a function correspondingto the user input of the determined type.
 2. The signal processingapparatus of claim 1, wherein the frequency detector comprises aplurality of vibration structures that sense vibration inputs and soundsof different frequency bands, and wherein each of the plurality ofvibration structures comprises a vibrator configured to vibrate based onthe frequency of the user input as the user input is received.
 3. Thesignal processing apparatus of claim 2, wherein each of the plurality ofvibration structures further comprises a vibration detector configuredto receive a vibration of the vibrator and generate an electrical signalcorresponding to the received vibration.
 4. The signal processingapparatus of claim 1, wherein the processor is further configured to:determine that the type of the received user input is the vibrationinput when the detected frequency is in a first frequency band; anddetermine that the type of the received user input is the user voicewhen the detected frequency is in a second frequency band.
 5. The signalprocessing apparatus of claim 4, wherein the processor is furtherconfigured to determine that the type of the received user input is theuser voice when the received user input lasts longer than a first lengthof time and the frequency of the received user input is in the secondfrequency band.
 6. The signal processing apparatus of claim 4, whereinthe processor is further configured to determine that the type of thereceived user input is the vibration input when the frequency of thereceived user input is in an overlapping frequency band that is in boththe first frequency band and the second frequency band while thereceived user input lasts.
 7. The signal processing apparatus of claim4, wherein the processor is further configured to determine that thetype of the received user input is the user voice when the frequency ofthe received user input is in both the first frequency band and in aportion of the second frequency band not overlapping the first frequencyband.
 8. The signal processing apparatus of claim 4, wherein the firstfrequency band corresponds to a resonance frequency band of a housing ofthe signal processing apparatus, and wherein the second frequency bandcorresponds to a voice grade.
 9. The signal processing apparatus ofclaim 4, wherein the vibration input comprises at least one of a tapinput, a swipe input, and a bone conduction input.
 10. The signalprocessing apparatus of claim 9, wherein the processor is furtherconfigured to: determine that the type of the received user input is thetap input when the frequency of the user input is in the first frequencyband while the received user input lasts, and a duration of the userinput is less than or equal to a first length of time, and determinethat the type of the user input is the swipe input or the boneconduction input when the duration of the user input exceeds the firstlength of time.
 11. The signal processing apparatus of claim 9, whereinthe processor is further configured to: determine that the type of thereceived user input is the tap input when the frequency of the userinput is in the first frequency band while the received user inputlasts, and a magnitude of an electrical signal generated by thefrequency detector, based on the vibration, is equal to or greater thana threshold level, and determine that the type of the user input is theswipe input or the bone conduction input when the magnitude of theelectrical signal is less than the threshold level.
 12. The signalprocessing apparatus of claim 1, wherein the processor is furtherconfigured to perform a first function corresponding to the user voiceand a second function corresponding to the vibration input when thereceived user input is determined to include both the user voice and thevibration input.
 13. The signal processing apparatus of claim 12,wherein the processor is further configured to identify a user who usesthe signal processing apparatus, based on a combination of the uservoice and the vibration input.
 14. The signal processing apparatus ofclaim 12, wherein the signal processing apparatus further comprises adisplay configured to output visual information, and wherein theprocessor is further configured to control the display to displayresults of performing the first function corresponding to the user voiceand the second function corresponding to the vibration input ondifferent regions on the display.
 15. The signal processing apparatus ofclaim 1, wherein the signal processing apparatus comprises an eyeglasswearable device, and wherein the frequency detector is arranged on aneyeglass frame of the eyeglass wearable device.
 16. A signal processingmethod comprising: receiving a user input comprising at least one of avibration input and a user voice; vibrating in response to the receiveduser input; detecting a frequency of the received user input, based onthe vibration; detecting a type of the received user input, based on thedetected frequency; and performing a function corresponding to the userinput of the determined type.
 17. The signal processing method of claim16, wherein the vibrating in response to the received user inputcomprises vibrating based on the frequency of the received user input,wherein the vibrating is performed by a plurality of vibrationstructures for sensing vibration inputs and sounds of differentfrequency bands.
 18. The signal processing method of claim 16, whereinthe determining of the type of the received user input comprisesdetermining that the type of the received user input is the vibrationinput when the detected frequency is in a first frequency band, anddetermining that the type of the received user input is the user voicewhen the detected frequency is in a second frequency band.
 19. Thesignal processing method of claim 16, wherein the performing of thefunction corresponding to the user input of the determined typecomprises performing a first function corresponding to the user voiceand a second function corresponding to the vibration input when thereceived user input is determined to include both the user voice and thevibration input.
 20. The signal processing method of claim 16, whereinthe performing of the function corresponding to the user input of thedetermined type comprises displaying results of performing a firstfunction corresponding to the user voice and a second functioncorresponding to the vibration input on different regions on a display.